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ABSTRACT 



Two features limit the value of legislatively mandated 
high-stakes tests such as Alaska's Benchmark and High School Graduation 
Qualifying Exam as accountability tools in the current standards -driven 
environment. First, the sheer numbers of tests administered have led to a 
reliance on multiple choice and short-answer questions, with only minimal use 
of more useful, performance-based approaches to assessment. This has left the 
harder- to-measure aspects of the standards in the background. Teachers are 
left to choose between teaching the full range of learning outcomes, or 
teaching to the test. The second problem is that when high- stakes tests are 
used to determine things such as grade-level promotion, eligibility for 
graduation, teacher reward/punishment, and school ranking, test-makers design 
the tests around legal def ensibility rather than educational considerations. 
When there are significant group variations in test performance, as in 
cross-cultural situations found in rural Alaska, it is important not to fall 
into the trap of blaming the victim and respond by intensifying the current 
curriculum, extending schooling, or sending students to boarding schools. The 
Alaska Rural Systemic Initiative (AKRSI ) has demonstrated that an education 
system with a strong foundation in the local culture produces positive 
effects in all indicators of school success, including test scores. AKRSI has 
developed cultural standards to assist schools that regard cultural 
considerations as an important part of the design of their educational 
programs. (TD) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



VOL. 5, ISSUE 4 
Sept/Oct 2000 



™ A newsletter of the Alaska Rural Systemic Initiative 

Alaska Federation of Natives ❖ University of Alaska <> National Science Foundation O Annenberg Rural Challenge 



oo 

o 

o 

m 



Q 

W 




by Ray Earnhardt , Angayuqaq Oscar Kawagley and Frank Hill , 

Alaska Rural Systemic Initiative 



i 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 
EDUCATIONAL RESOURCES INFORMATION 
y, CENTER (ERIC) 

jJrThis document has been reproduced as 
received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 



"PERMISSION TO REPRODUCE THIS 
MATBB^L HAS BEEN GRANTED BY 



TEBIAL 



gy| 






• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 

- - _ ^ 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)/’ 



O'- 




sc 




BEST COPY AVAILABLE 



2 



OAA. 




.(♦x»x»x#x^x»x»x»x^x»x^x^xtXf? t 






by Ray Barnhardt, Angayuqaq Oscar Kawagley and Frank Hill, 

Alaska Rural Systemic Initiative 



W ith the release of the first Benchmark and High School 
Graduation Qualifying Exam scores this fall, educators 
throughout Alaska have been convening to address the many 
issues that are raised by these new checkpoints on the educa- 
tional landscape. 



Debates are already underway on 
ways to interpret the results and de- 
velop appropriate responses, given 
the predictable differences in perfor- 



mance among various students and 
schools. At the heart of these debates 
are concerns over the use (or misuse) 
of the test results to make critical 



judgements about students, teachers 
and schools in ways that attempt to 
reduce complex school performance 
issues down to a few simplistic 
variables. 

We need look no further than the 
latest editions of Education Week , Phi 
Delta Kappan or Educational Re- 
searcher to see that these debates are 
occurring on a national scale and that 
Alaska is not alone in venturing out 
(continued on next page) 
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Harriet Nungasak, Alicia Kanayurak, George Olanna (instructor), 
Kimberly Rychnovsky and Donald Tritt work on their science projects at 
the ANSES Science-Culture Camp held at Gaaleeya Spirit Camp July 11- 
25, 2000. Skills gained at science and cultural camps are brought into the 
classroom and utilized throughout the school year. See “A1SES Corner”, 
page 6. 
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( continued from front page) 
into uncharted waters in the name of 
school accountability. Hopefully we 
can lear-n-from other peoples-mistakes, 
and by doing a few things right, maybe 
others can learn from our successes. 
However, this will require taking a 
long-term perspective on the many 
issues involved and not expecting to 
find a silver bullet that will produce 
instant solutions to long-standing 
complex problems. 

First of all, we must recognize the 
practical limits of the tests themselves. 
As diagnostic tools coupled with other 
related indicators of ability and per- 
formance, tests that are properly de- 
signed, flexibly administered and 
judiciously interpreted can provide 
valuable information to guide educa- 
tional decision-making. However, 
there are two features of these legisla- 
tively mandated high-stakes tests that 
inhibit their educational value and 
thus make it necessary to exercise 
considerable caution in their use as 
accountability tools in the current 
standards-driven environment. 

Since the tests are mandated for all 
studentsatfourgrade levels, the sheer 
number and frequency of the testing 
introduces a major time and cost fac- 
tor. As a result, the design of the tests 
tends to rely on approaches that are 
simpler and cheaper to administer and 
score (i.e., multiple choice and short- 
answer questions) with only minimal 
use of the more costly, but flexible, 
culturally adaptable and education- 
ally useful performance-based 
approaches to assessment. Unfortu- 
nately, this emphasis on ease of ad- 
ministration has also narrowed the 
selection of which content standards 
count and which ones don't, leaving 
the harder-to-measure aspects of the 
standards in the background. 

As a result, teachers (and districts) 
are caught in the dilemma of aligning 
their teaching and curriculum with 
the full range of learning outcomes 
outlined in the standards or narrow- 



ing their lessons down to that which 
is measured on the tests (see Education 
Week , July 7, 2000, p.l at 

www.edweek.org for a more detailed 
discussion of this issue). In this re- 
gard, the current testing system can 
be seen as working against the imple- 
mentation of the standards-based 
school reform efforts with which it 
was originally associated. A true 
standards-based educational system 
requires a much broader approach to 
assessment than current resources 
allow. 

The second feature that reduces 
the educational value of high-stakes 
testing is its intended use in making 
critical decisions that can adversely 
impact people'slivesand careers(e.g., 
grade-level promotion, eligibility for 
graduation, teacher reward/punish- 
ment and school rating/ranking.) 
When used for such purposes, the 
tests themselves tend to revert to those 
measures that the test-makers (in 
Alaska's case, CBT/McGraw-Hill) can 
defend in court when challenged by 
those affected. Consequently, we see 
a heavy emphasis on standardization 
(in both content and administration), 
whereby many important aspects of 
the content standards that require 
local adaptation or are not easily mea- 
sured are set aside in favor of those 
items and testing practices that meet 
the test of "legal defensibility ." So we 
should not be surprised when we run 
into problems with a testing system 
that has been constructed around le- 
gal and political, rather than educa- 
tional considerations. 
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For better or worse, the Alaska 
Benchmark and High School Gradua- 
tion Qualifying Exams are a reality 
and it is our professional responsibil- 
ity to do what we can to minimize 
their negative effects and to maximize 
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their potential benefits. Most critical 
in that regard is the need to examine 
the issues that emerge in the broadest 
context available to us and not to use 
the results to promote simplistic, short- 
term solutions to long-term, complex 
problems. Nor should we fall into the 
trap of "blaming the victim" (i.e., the 
student) when there are significant 
group variations in academic perfor- 
mance. This is especially true in a 
cross-cultural setting such as rural 
Alaska, where we have a long history 
of repetitious unsuccessful educa- 
tional experimentation on students 
while ignoring the well-documented 
source of many of the problems — that 
is the persistent cultural gulf between 
teachersand students, school and com- 
munity. 

Based on the experiences in other 
states and the rife speculation under- 
way here in Alaska, we can expect 
several things to happen over the next 
few months. The initial responses to 
the release of the test results are likely 
to point to two factors to explain 
differential performance between stu- 
dents and schools — low teacher ex- 
pectations and lack of opportunity to 
learn — each of which will lead to pre- 
dictable forms of remediation. 

Under the banner of "all students 
can learn to high standards," teachers 
will be admonished to teach harder 
and more of whatever it is that stu- 
dents are determined by the tests as 
lacking. While this may seem logical 
on the surface, it ignores the possibil- 
ity that the real issue may not be low 
expectations at all (though certainly 
that does exist) and that "more of the 
same" may exacerbate the problem 
by producing higher dropout rates 
rather than addressing the more fun- 
damental issue of lack-of-fit between 
what we teach, how we teach it and 
the context in which it is taught. 
Intensifying the current curriculum 
and extending schooling into the 
weekend or summer also ignores the 
inherent limitations to school improve- 
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ment in rural Alaska that result from 
having to import teachers and admin- 
istrators from outside for whom the 
village setting isa foreign and inevita- 
bly temporary home. 

The second issue of making sure 
students have had the opportunity to 
learn the subject matter on which 
they are being tested is more readily 
identifiable as a problem, but no less 
complicated (and expensive) in pro- 
ducing a solution. If a small rural 
school is not offering the level of math- 
ematics instruction that students need 
to pass the exam, the solution is not to 
send the students elsewhere for school- 
ing. To assume that a boarding school 
(as some legislators are suggesting) 
can make up for the limitations of a 
village high school ignores the fact 
that a well-rounded education con- 
sists of much more than just the sub- 
ject matter that is taught in school. It 
also ignores the negative impact that 
taking students out of their home has 
on the family, the community and the 
student's own future role as a parent 
and contributing member of society. 
There is nothing taught in a boarding 
school that can't be taught cheaper 
and more effectively in a village school 
linked together with other village 
schools in a web of rich and extensive 
learning opportunities. Furthermore, 
there are many important things that 
are learned at home in a village setting 
that cannot be taught in a boarding 
school. Boarding schools may be jus- 
tified as an optional alternative pro- 
gram for selected students, but not as 
a substitute for village schools. 

When providing "opportunities to 
learn," we need to consider all aspects 
of a child's upbringing and prepare 
them in such a way that they can 
"become responsible, capable and 
whole human beings in the process" 
(see Alaska Standards for Culturally 
Responsive Schools). When we do so, 
the issues associated with benchmark 
and qualifying exams will take care of 
themselves. How then do we go about 



The Alaska 
Standards 
for 

Culturally- 
Responsive 
Schools is 
available 
from: 

Alaska Native 
Knowledge Network 
P.O. Box 756730 
Fairbanks, AK 99775-6730 

It can also be found on the 
ANKN website: 

http://www.ankn.uaf.edu. 



this with some degree of confidence 
that we will achieve the outcome we 
seek — graduates capable of function- 
ing as responsible adults, including 
passing state exams? 
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For the past five years, the Alaska 
Rural Systemic Initiative has been 
working intensively with 20 of the 48 
rural school districts in the state to 
implement a series of initiatives that 
are intended to "systematically docu- 
ment the indigenous knowledge sys- 
tems of Alaska Native people and 
develop educational policiesand prac- 
tices that effectively integrate indig- 
enous and western knowledge 
through a renewed educational sys- 
tem." The assumption behind the 
AKRSI reform strategy is that if we 
coordinate our efforts and resources 
across all aspects of the education 
system and address the issues in a 
(continued on next page) 
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(continued from previous page) 
focused, statewide manner, perhaps 
better headway will be realized. Two 
outcomes of this work are worthy of 
consideration as schools review the 
results of the state tests and ponder 
their next steps. 

First of all, building an education 
system with a strong foundation in 
the local culture appears to produce 
positive effects in all indicators of 
school success, including dropout 



rates, college attendance, parent in- 
volvement, grade-point averages and 
standardized achievement test scores. 
With regard to student achievement, 
using the eighth-grade CAT-5 math 
test scores as an impact indicator for 
the first four years of implementation 
of the AKRSI school reform initiatives 
in the 20 participating school districts 
(which have historically had the low- 
est student achievement levels in the 
state), there has been a differential 
gain of 5.9% points in the number of 
students who are performing in the 
top quartile for AKRSI partner schools 
over non-AKRSI rural schools. AKRSI 
schools gained 6.9% points in the 
upper quartile compared to a 1.0% 
point gain for non-AKRSI schools, 



with a corresponding decrease in the 
lower quartile. With AKRSI districts 
now producing 24.3% of their stu- 
dents testing in the upper quartile, 
they are only 0.7% point below the 
national average. In other words, 
through strong place-based education 
initiatives, the AKRSI schools are clos- 
ing the achievement gap with the 
non-AKRSI schools. The following 
graph illustrates the gains on a year- 
by-year basis: 



In reviewing this data (drawn from 
the state summary of the school dis- 
trict report cards), it is clear that some- 
thing has been going on in the 20 
AKRSI school districts that is produc- 
ing a slow but steady gain in the 
standardized test scores (along with 
all the other indicators we have been 
tracking.) So just what is it that is 
producing these results? Since the 
gains are widespread across all cul- 
tural regions and the scores show con- 
sistent improvement over each of the 
four years, they clearly are not a func- 
tion of one particular curricular or 
pedagogical initiative, nor are they 
limited to AKRSI-sponsored activi- 
ties. The best summary of what it is 
that has produced these results can be 



found in the Alaska Standards for 
Culturally Responsive Schools . 

These ''cultural standards" were 
compiled by educators from through- 
out the state as an outgrowth of the 
work that was initiated through the 
Alaska Rural Systemic Initiative and 
implemented in varying degrees by 
the participating schools. As such, 
when coupled with the impact data 
summarized here, they provide some 
concrete guidelines for schools and 
communities to consider as they con- 
struct school improvement plans 
aimed at producing more effective 
educational programs for the students 
in their care. We now have strong 
evidence that when we make a dili- 
gent and persistent effort to forge a 
strong cultural fit between what we 
teach, how we teach and the context 
in which we teach, we can produce 
successful, well-rounded graduates 
who are also capable of producing 
satisfactory test scores. 

The AKRSI staff are currently 
working with the Alaska Department 
of Education and Early Development 
to provide assistance to schools for 
whom cultural considerations play an 
important part in the design of their 
educational programs. Alaska Native 
educators, including Elders, are an 
important resource that all schools 
need to draw upon to make sure that 
our responses to the results of the 
Alaska Benchmark and High School 
Graduation Qualifying Exams go 
beyond Band-Aid solutions and lead 
to long-term improvement of our 
education systems. The future of our 
state depends on it. Curricular re- 
sources and technical assistance for 
such efforts are available through 
the regional Native Educator Assoc- 
iations, as well as the Alaska Native 
Knowledge Network web site at 
www.ankn.uaf.edu. ^ 
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Percentage of Students on Top Quartile on CATS , 1996—1999 
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□ Non-AKRSI 
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